尽管进行了数十年的研究，但现有的导航系统在野外部署时仍然面临现实世界中的挑战，例如在混乱的家庭环境或人类占领的公共场所中。为了解决这个问题，我们提出了一类新的隐式控制政策，将模仿学习的好处与模型预测控制（MPC）的系统约束的强大处理结合在一起。我们的方法称为Performer-MPC，使用了通过表演者提供的视觉上下文嵌入的学习成本函数（一种低级隐式意见变压器）。我们共同训练成本函数并构建依靠它的控制器，有效地端到端解决相应的双层优化问题。我们表明，由此产生的策略通过利用一些在不同挑战的现实世界情景中利用一些专家演示来提高标准MPC绩效。与标准的MPC政策相比，表演者MPC在混乱的环境中实现了40％的目标，而在人类浏览时，社交指标的目标> 65％。

在我们最近在加纳被动饮食监测的饮食评估现场研究中，我们收集了超过25万件野外图像。该数据集是一种持续的努力，旨在通过被动监控摄像头技术在低收入和中等收入国家中准确测量单个食物和营养摄入量。目前的数据集涉及加纳农村地区和城市地区的20个家庭（74个受试者），研究中使用了两种不同类型的可穿戴摄像机。一旦开始，可穿戴摄像机会不断捕获受试者的活动，该活动会产生大量的数据，以便在进行分析之前清洁和注释。为了简化数据后处理和注释任务，我们提出了一个新颖的自学学习框架，以将大量以自我为中心的图像聚集到单独的事件中。每个事件都由一系列时间连续和上下文相似的图像组成。通过将图像聚集到单独的事件中，注释者和营养师可以更有效地检查和分析数据，并促进随后的饮食评估过程。在带有地面真实标签的固定测试套装上验证，拟议的框架在聚集质量和分类准确性方面优于基准。

人的大脑可以毫不费力地识别和定位对象，而基于激光雷达点云的当前3D对象检测方法仍然报告了较低的性能，以检测闭塞和远处的对象：点云的外观由于遮挡而变化很大，并且在沿线的固有差异沿点固有差异变化。传感器的距离。因此，设计功能表示对此类点云至关重要。受到人类联想识别的启发，我们提出了一个新颖的3D检测框架，该框架通过域的适应来使对象完整特征。我们弥合感知域之间的差距，其中特征是从具有亚最佳表示的真实场景中得出的，以及概念域，其中功能是从由不批准对象组成的增强场景中提取的，并具有丰富的详细信息。研究了一种可行的方法，可以在没有外部数据集的情况下构建概念场景。我们进一步介绍了一个基于注意力的重新加权模块，该模块可适应地增强更翔实区域的特征。该网络的功能增强能力将被利用，而无需在推理过程中引入额外的成本，这是各种3D检测框架中的插件。我们以准确性和速度都在Kitti 3D检测基准上实现了新的最先进性能。关于Nuscenes和Waymo数据集的实验也验证了我们方法的多功能性。

非平行的多与众不同的语音转换仍然是一项有趣但具有挑战性的语音处理任务。最近，基于有条件的自动编码器的方法AutoVC通过使用信息限制的瓶颈来删除说话者身份和语音内容，从而实现了出色的转换结果。但是，由于纯粹的自动编码器训练方法，很难评估内容和说话者身份的分离效果。在本文中，一个新颖的语音转换框架，名为$ \ boldsymbol t $ ext $ \ boldsymbol g $ uided $ \ boldsymbol a $ utovc（tgavc），提议更有效地将内容和音色与语音分开，其中预期的内容嵌入其中根据文本转录生产的旨在指导语音内容的提取。此外，对对抗性训练将用于消除从语音中提取的估计内容中的说话者身份信息。在预期内容嵌入和对抗培训的指导下，对内容编码器进行了培训，以从语音中提取嵌入说话者的内容。 Aishell-3数据集的实验表明，所提出的模型在自然性和转换语音的相似性方面优于AUTOVC。

Weakly-supervised object localization aims to indicate the category as well as the scope of an object in an image given only the image-level labels. Most of the existing works are based on Class Activation Mapping (CAM) and endeavor to enlarge the discriminative area inside the activation map to perceive the whole object, yet ignore the co-occurrence confounder of the object and context (e.g., fish and water), which makes the model inspection hard to distinguish object boundaries. Besides, the use of CAM also brings a dilemma problem that the classification and localization always suffer from a performance gap and can not reach their highest accuracy simultaneously. In this paper, we propose a casual knowledge distillation method, dubbed KD-CI-CAM, to address these two under-explored issues in one go. More specifically, we tackle the co-occurrence context confounder problem via causal intervention (CI), which explores the causalities among image features, contexts, and categories to eliminate the biased object-context entanglement in the class activation maps. Based on the de-biased object feature, we additionally propose a multi-teacher causal distillation framework to balance the absorption of classification knowledge and localization knowledge during model training. Extensive experiments on several benchmarks demonstrate the effectiveness of KD-CI-CAM in learning clear object boundaries from confounding contexts and addressing the dilemma problem between classification and localization performance.

An increasing number of public datasets have shown a marked clinical impact on assessing anatomical structures. However, each of the datasets is small, partially labeled, and rarely investigates severe tumor subjects. Moreover, current models are limited to segmenting specific organs/tumors, which can not be extended to novel domains and classes. To tackle these limitations, we introduce embedding learned from Contrastive Language-Image Pre-training (CLIP) to segmentation models, dubbed the CLIP-Driven Universal Model. The Universal Model can better segment 25 organs and 6 types of tumors by exploiting the semantic relationship between abdominal structures. The model is developed from an assembly of 14 datasets with 3,410 CT scans and evaluated on 6,162 external CT scans from 3 datasets. We rank first on the public leaderboard of the Medical Segmentation Decathlon (MSD) and achieve the state-of-the-art results on Beyond The Cranial Vault (BTCV). Compared with dataset-specific models, the Universal Model is computationally more efficient (6x faster), generalizes better to CT scans from varying sites, and shows stronger transfer learning performance on novel tasks. The design of CLIP embedding enables the Universal Model to be easily extended to new classes without catastrophically forgetting the previously learned classes.

In this work, we tackle two vital tasks in automated driving systems, i.e., driver intent prediction and risk object identification from egocentric images. Mainly, we investigate the question: what would be good road scene-level representations for these two tasks? We contend that a scene-level representation must capture higher-level semantic and geometric representations of traffic scenes around ego-vehicle while performing actions to their destinations. To this end, we introduce the representation of semantic regions, which are areas where ego-vehicles visit while taking an afforded action (e.g., left-turn at 4-way intersections). We propose to learn scene-level representations via a novel semantic region prediction task and an automatic semantic region labeling algorithm. Extensive evaluations are conducted on the HDD and nuScenes datasets, and the learned representations lead to state-of-the-art performance for driver intention prediction and risk object identification.

New architecture GPUs like A100 are now equipped with multi-instance GPU (MIG) technology, which allows the GPU to be partitioned into multiple small, isolated instances. This technology provides more flexibility for users to support both deep learning training and inference workloads, but efficiently utilizing it can still be challenging. The vision of this paper is to provide a more comprehensive and practical benchmark study for MIG in order to eliminate the need for tedious manual benchmarking and tuning efforts. To achieve this vision, the paper presents MIGPerf, an open-source tool that streamlines the benchmark study for MIG. Using MIGPerf, the authors conduct a series of experiments, including deep learning training and inference characterization on MIG, GPU sharing characterization, and framework compatibility with MIG. The results of these experiments provide new insights and guidance for users to effectively employ MIG, and lay the foundation for further research on the orchestration of hybrid training and inference workloads on MIGs. The code and results are released on https://github.com/MLSysOps/MIGProfiler. This work is still in progress and more results will be published soon.

There are multiple scales of abstraction from which we can describe the same image, depending on whether we are focusing on fine-grained details or a more global attribute of the image. In brain mapping, learning to automatically parse images to build representations of both small-scale features (e.g., the presence of cells or blood vessels) and global properties of an image (e.g., which brain region the image comes from) is a crucial and open challenge. However, most existing datasets and benchmarks for neuroanatomy consider only a single downstream task at a time. To bridge this gap, we introduce a new dataset, annotations, and multiple downstream tasks that provide diverse ways to readout information about brain structure and architecture from the same image. Our multi-task neuroimaging benchmark (MTNeuro) is built on volumetric, micrometer-resolution X-ray microtomography images spanning a large thalamocortical section of mouse brain, encompassing multiple cortical and subcortical regions. We generated a number of different prediction challenges and evaluated several supervised and self-supervised models for brain-region prediction and pixel-level semantic segmentation of microstructures. Our experiments not only highlight the rich heterogeneity of this dataset, but also provide insights into how self-supervised approaches can be used to learn representations that capture multiple attributes of a single image and perform well on a variety of downstream tasks. Datasets, code, and pre-trained baseline models are provided at: https://mtneuro.github.io/ .

Designing better deep networks and better reinforcement learning (RL) algorithms are both important for deep RL. This work focuses on the former. Previous methods build the network with several modules like CNN, LSTM and Attention. Recent methods combine the Transformer with these modules for better performance. However, it requires tedious optimization skills to train a network composed of mixed modules, making these methods inconvenient to be used in practice. In this paper, we propose to design \emph{pure Transformer-based networks} for deep RL, aiming at providing off-the-shelf backbones for both the online and offline settings. Specifically, the Transformer in Transformer (TIT) backbone is proposed, which cascades two Transformers in a very natural way: the inner one is used to process a single observation, while the outer one is responsible for processing the observation history; combining both is expected to extract spatial-temporal representations for good decision-making. Experiments show that TIT can achieve satisfactory performance in different settings, consistently.